Presence detection and uses thereof

ABSTRACT

A computer-implemented method for detecting a person in a room using a sensor, such as a radar sensor or depth cameras is provided. The method can be used to detect sleeping persons in a publically accessible location to prevent loitering. Furthermore, the method can be used to identify which rooms amongst a plurality of rooms are occupied, for example changing rooms.

RELATED APPLICATIONS

This application is a continuation of PCT International Application No.PCT/CA2017051179 filed on Oct. 3, 2017. This application also claims thebenefit of U.S. Provisional Patent Application No. 62/404,124 filed onOct. 4, 2016, and U.S. Provisional Patent Application No. 62/477,195filed on Mar. 27, 2017. All of the above-mentioned applications arehereby incorporated by reference in their entireties.

FIELD

The present subject-matter relates to a sensor system, camera, and asystem integrating the camera and sensor system configured to detect aperson.

BACKGROUND

A camera may be used to acquire information about a place or an object.The information is visual image data generated by the cameracorresponding to the scene falling with the field of view of the camera.A sensor system, such as a radar or depth device, may be used to acquireother information about a place or an object.

Some currently available security systems include means of detectingloitering, often based on tracking the movement of an object within afixed area for a period of time. However typical video analytics systemsfail to detect an immobile person, who simply becomes part of thebackground image. Security systems available are unable to detectsleeping persons as they do not move sufficiently to register on acamera system; instead they are treated like immobile objects. Thisposes problems in situations in which the object being monitored is atrest and moving only minimally. An example of such a situation occurs incertain areas, such as ATM vestibules, which are attractive andrelatively warm areas for sleeping compared to the outdoors.

Currently banks and other operators of publicly accessible areas, forexample ATM vestibules, have difficulty detecting and moving sleepingpersons from their property. The presence of persons loitering orsleeping in a vestibule may discourage customers from entering thevestibule.

Another challenge for video cameras is detecting objects or persons inmultiple rooms, or in situations where privacy needs to be protected,for example changing rooms or showers.

SUMMARY

The embodiments described herein provide in one aspect a combined videosurveillance system including a sensor system and a camera device. Thecamera device may include at least one camera module operable forcapturing a scene corresponding to a field of view of the camera and forgenerating image data of the captured scene, a transceiver operable forreceiving the access data transmitted from the battery-powered physicalaccess control device, a networking module operable for transmitting theimage data and access data to an external networked device, a wirelesspower transmitter operable for transmitting power wirelessly to thewireless power receiver of the access control device, and at least onepower supply operable for receiving power over a wired connection andsupplying power to the camera, the transceiver, the network module andthe wireless power transmitter.

The embodiments described herein provide another aspect of a cameradevice. The camera device includes at least one camera module operablefor capturing a scene corresponding to a field of view of the cameramodule and for generating image data of the captured scene, at least onewireless power transmitter operable for transmitting wireless power to abattery-powered external sensing device, a transceiver operable forreceiving from the external sensing device sensed data pertaining to atleast one condition sensed by the external sensing device, a networkingmodule operable for transmitting the image data and data pertaining tothe at least one sensed condition to an external networked device, andat least one power supply operable for receiving power over a wiredconnection and supplying power to the camera module, the transceiver,the networking module and the wireless power transmitter.

According to some example embodiments, the networking module transmitsthe image data and the access data to the external networked device overthe wired connection.

According to some example embodiments, the camera device furtherincludes a video analytics module operable for performing videoanalytics on the image data and determining an occurrence of a videoanalytics event based on a combination of one or more results of theperformed video analytics and access data received from the accesscontrol device.

According to some example embodiments, the transceiver of the cameradevice is further operable for receiving information pertaining to abattery status of the at least one battery of the access control deviceand the wireless power transmitter is operable for adjusting thetransmission of wireless power to the access control device based on thereceived information pertaining to the battery status.

According to some example embodiments, an effective powered space of thewireless power transmitter of the camera device substantially overlapswith the field of view of the camera module.

According to some example embodiments, the camera module is pivotableand wherein the wireless power transmitter pivots with the camera moduleto maintain overlap of the power coverage cone with the field of view.

According to some example embodiments, the camera device comprises aplurality of wireless power transmitters, each transmitter beingoperable for directionally transmitting wireless power.

According to some example embodiments, at least one of the plurality ofwireless power transmitters is pivotable to change the space occupied byits power coverage cone.

According to some example embodiments, the sensor system includes aradar system configured to determine the presence of a person.

According to some example embodiments, the sensor system includes aradar system or a depth camera, and the sensor system, on determinationof the presence of a person, provides an alert.

According to some example embodiments, a method to detect a personloitering in an area under surveillance is provided, including:generating a model of the background of the area under surveillance fora depth sensor; obtaining data maps of the area under surveillance overa period of time from the depth sensor; comparing the data maps with themodel of the background to generate foreground frames over the period oftime; and detecting a person loitering in the area under surveillancefrom analyzing the foreground frames over the period of time.

According to some example embodiments, a security system for detecting aperson loitering in an area under surveillance is provided, including adepth sensor configured to monitor the area under surveillance and aprocessor coupled to the depth sensor and configured to: generate amodel of the background of the area under surveillance for the depthsensor; obtain data maps of the area under surveillance over a period oftime from the depth sensor; compare the data maps with the model of thebackground to generate foreground frames over the period of time; anddetect a person loitering in the area under surveillance from analyzingthe foreground frames over the period of time.

According to some example embodiments, a security system for detecting aperson loitering in an area under surveillance is provided, including adepth sensor configured to monitor the area under surveillance; a radartransceiver configured to monitor the area under surveillance, the radarsystem configured to transmit radar signals to and receive radar signalsfrom the area under surveillance; a processor coupled to the depthsensor and configured to: generate a model of the background of the areaunder surveillance for the depth sensor; obtain data maps of the areaunder surveillance over a period of time from the depth sensor; comparethe data maps with the model of the background to generate foregroundframes over the period of time; and detect a person loitering in thearea under surveillance from analyzing the foreground frames over theperiod of time; and the processor coupled to the radar transceiver andconfigured to: determine if the radar signals indicate a presence of aperson in the area under surveillance; on determination of the presenceof the immobile person, determine if the immobile person is present inthe area under surveillance for a predetermined time period; and ondetermination of the presence of the person for a time period greaterthan the predetermined time period, providing instructions to cause analert.

According to some example embodiments, a computer-implemented method fordetecting a person in a publicly accessible location is provided,including: providing a radar system positioned to monitor the publiclyaccessible location, the radar system configured to transmit radarsignals to and receive radar signals from the location; determining ifthe radar signals indicate a presence of a person in the location; andon determination of the presence of the person, causing an alert.

According to some example embodiments, a method for controllingloitering in an unsecured location is provided, including: providing aradar transceiver in a secured location, the radar transceiverconfigured to transmit radar signals to and receive radar signals fromthe unsecured location; determining if the received radar signalsindicate a presence of a first person in the unsecured location; if thefirst person is present, determining if the first person is present inthe unsecured location for a predetermined time period; and if the firstperson is present in the unsecured time period for a time greater thanthe predetermined time period, providing an alert, report or alarm.

According to some example embodiments, a physical security system isprovided, including: a radar transceiver configured to monitor apublicly accessible location, the radar system configured to transmitradar signals to and receive radar signals from the publicly accessiblelocation; a processor coupled to the radar transceiver and configuredto: determine if the radar signals indicate a presence of a person inthe publicly accessible location; on determination of the presence ofthe immobile person, determine if the immobile person is present in theunsecured location for a predetermined time period; on determination ofthe presence of the person for a time period greater than thepredetermined time period, providing instructions to cause an alert.

According to some example embodiments, a physical security system formonitoring a plurality of rooms is provided, including: a radartransceiver configured to monitor the plurality of rooms, the radartransceiver configured to transmit radar signals to and receive radarsignals from each room in the plurality of rooms; a processorcommunicatively coupled to the transceiver and configured to determineif each of said rooms is occupied.

The processor may be further configured to determine if a detectedoccupant is asleep or in distress, or to determine the length of time inwhich the occupant has been present in a room. The rooms may be cells orchanging rooms, or may contain one or more animals.

According to some sample embodiments, a method for controlling loiteringin a room within a plurality of rooms is provided, including: providinga radar transceiver, the radar transceiver configured to transmit radarsignals to and receive radar signals from each room within the pluralityof rooms; determining if the received radar signals indicate a presenceof a first person in the room; if the first person is present,determining if the first person is present in the room for apredetermined time period; and if the first person is present in theroom for a time greater than the predetermined time period, providing analert, report or alarm.

According to some sample embodiments, a computer-implemented method fordetecting a person in a room amongst a plurality of rooms is provided,including: providing a radar system positioned to monitor each roomamongst the plurality of rooms, the radar system configured to transmitradar signals to and receive radar signals from each room; determiningif the radar signals indicate a presence of a person in one or more ofthe rooms amongst the plurality of rooms; and on determination of thepresence of the person, causing the generation of a report comprisingidentification of the rooms in which the presence of a person wasindicated.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description refers to the following figures, in which:

FIG. 1 illustrates a block diagram of a camera device according to anexample embodiment;

FIG. 2 illustrates a block diagram of a combined system according to anexample embodiment having an example camera device and a sensor system;

FIG. 3A illustrates a block diagram of a combined system according to analternative example embodiment having an example camera device and aradar system;

FIG. 3B illustrates a block diagram of a combined system according toanother alternative example embodiment having an example camera deviceand a radar system;

FIG. 4A illustrates a block diagram of a camera device according to analternative example embodiment;

FIG. 4B illustrates a block diagram of a schematic diagram of an exampledeployment of the alternative camera device according to one exampleembodiment;

FIG. 5 illustrates a block diagram of connected devices of a videosurveillance system according to one example embodiment;

FIG. 6 illustrates a schematic diagram of an example deployment of acamera device, a sensor system, and an ATM;

FIG. 7 illustrates a schematic diagram of an example deployment of acamera device and radar device;

FIG. 8 illustrates a block diagram of a radar device with a cameradevice according to an example embodiment;

FIG. 9 illustrates a block diagram of a radar device according to anexample embodiment;

FIG. 10 illustrates a radar device according to an example embodiment;

FIG. 11A illustrates a schematic diagram of an example deployment of aradar device according to an example embodiment;

FIG. 11B illustrates a schematic diagram of an example deployment of aradar device according to an alternative example deployment;

FIG. 12 illustrates an installation of two 3D cameras on the ceiling ofa room according to an example embodiment;

FIG. 13 illustrates a mounting for two 3D cameras for a larger field ofview, according to an example embodiment;

FIG. 14 illustrates an alternative mounting for two 3D cameras for alarger field of view, according to an example embodiment;

FIG. 15 illustrates example images from the installation of FIG. 12;

FIG. 16 illustrates additional example images from the installation ofFIG. 12;

FIG. 17 illustrates additional example images from the installation ofFIG. 12 with a person;

FIG. 18 illustrates additional example images from the installation ofFIG. 12 with a person;

FIG. 19 illustrates a flowchart of image processing of the installationof FIG. 12 according to an example embodiment;

FIG. 20 illustrates another installation of a radar sensor and a depthsensor on the ceiling of a room according to an example embodiment;

FIG. 21 illustrates a flowchart of a process by which a radar systemdetermines loitering according to an example embodiment;

FIG. 22 illustrates a deployment of a radar system for monitoring aplurality of changing rooms according to an example embodiment;

FIG. 23 illustrates a display generated by a radar system monitoring aplurality of changing rooms according to an example embodiment;

FIG. 24 illustrates a deployment of a radar system for monitoring aplurality of prison cells according to an example embodiment; and

FIG. 25 illustrates a display generated by a radar system monitoring aplurality of prison cells, according to an example embodiment.

It will be appreciated that for simplicity and clarity of illustration,elements shown in the figures have not necessarily been drawn to scale.For example, the dimensions of some of the elements may be exaggeratedrelative to other elements for clarity.

Furthermore, where considered appropriate, reference numerals may berepeated among the figures to indicate corresponding or analogouselements.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

Directional terms such as “top”, “bottom”, “upwards”, “downwards”,“vertically”, and “laterally” are used in the following description forthe purpose of providing relative reference only, and are not intendedto suggest any limitations on how any article is to be positioned duringuse, or to be mounted in an assembly or relative to an environment.Additionally, the term “couple” and variants of it such as “coupled”,“couples”, and “coupling” as used in this description is intended toinclude indirect and direct connections unless otherwise indicated. Forexample, if a first device is coupled to a second device, that couplingmay be through a direct connection or through an indirect connection viaother devices and connections. Similarly, if the first device iscommunicatively coupled to the second device, communication may bethrough a direct connection or through an indirect connection via otherdevices and connections.

The terms “an aspect”, “an embodiment”, “embodiment”, “embodiments”,“the embodiment”, “the embodiments”, “one or more embodiments”, “someembodiments”, “certain embodiments”, “one embodiment”, “anotherembodiment” and the like mean “one or more (but not all) embodiments ofthe disclosed invention(s)”, unless expressly specified otherwise. Areference to “another embodiment” or “another aspect” in describing anembodiment does not imply that the referenced embodiment is mutuallyexclusive with another embodiment (e.g., an embodiment described beforethe referenced embodiment), unless expressly specified otherwise.

The terms “including”, “comprising” and variations thereof mean“including but not limited to”, unless expressly specified otherwise.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

The term “plurality” means “two or more”, unless expressly specifiedotherwise. The term “herein” means “in the present application,including anything which may be incorporated by reference”, unlessexpressly specified otherwise.

The term “e.g.” and like terms mean “for example”, and thus does notlimit the term or phrase it explains.

The term “respective” and like terms mean “taken individually”. Thus iftwo or more things have “respective” characteristics, then each suchthing has its own characteristic, and these characteristics can bedifferent from each other but need not be. For example, the phrase “eachof two machines has a respective function” means that the first suchmachine has a function and the second such machine has a function aswell. The function of the first machine may or may not be the same asthe function of the second machine.

Where two or more terms or phrases are synonymous (e.g., because of anexplicit statement that the terms or phrases are synonymous), instancesof one such term/phrase does not mean instances of another suchterm/phrase must have a different meaning. For example, where astatement renders the meaning of “including” to be synonymous with“including but not limited to”, the mere usage of the phrase “includingbut not limited to” does not mean that the term “including” meanssomething other than “including but not limited to”.

Neither the Title (set forth at the beginning of the first page of thepresent application) nor the Abstract (set forth at the end of thepresent application) is to be taken as limiting in any way as the scopeof the disclosed invention(s). An Abstract has been included in thisapplication merely because an Abstract of not more than 150 words isrequired under 37 C.F.R. Section 1.72(b) or similar law in otherjurisdictions. The title of the present application and headings ofsections provided in the present application are for convenience only,and are not to be taken as limiting the disclosure in any way.

Numerous embodiments are described in the present application, and arepresented for illustrative purposes only. The described embodiments arenot, and are not intended to be, limiting in any sense. The presentlydisclosed aspect(s) are widely applicable to numerous embodiments, as isreadily apparent from the disclosure. One of ordinary skill in the artwill recognize that the disclosed aspect(s) may be practiced withvarious modifications and alterations, such as structural and logicalmodifications. Although particular features of the disclosed aspect(s)may be described with reference to one or more particular embodimentsand/or drawings, it should be understood that such features are notlimited to usage in the one or more particular embodiments or drawingswith reference to which they are described, unless expressly specifiedotherwise.

No embodiment of method steps or product elements described in thepresent application is essential or is coextensive, except where it iseither expressly stated to be so in this specification or expresslyrecited in a claim.

“Battery” herein refers to not only a device in which chemical energy isconverted into electricity and used as a source of power, it also refersto any alternatively suitable energy storage devices such as, forexample, a capacitor of suitable size and construction.

“Image data” herein refers to data produced by a camera device and thatrepresents images captured by the camera device. The image data mayinclude a plurality of sequential image frames, which together form avideo captured by the camera device. Each image frame may be representedby a matrix of pixels, each pixel having a pixel image value. Forexample, the pixel image value may be a numerical value on grayscale(e.g. 0 to 255) or a plurality of numerical values for colored images.Examples of color spaces used to represent pixel image values in imagedata include RGB, YUV, CYKM, YCbCr 4:2:2, YCbCr 4:2:0 images. It will beunderstood that “image data” as used herein can refer to “raw” imagedata produced by the camera device and/or to image data that hasundergone some form of processing. It will be further understood that“image data” may refer to image data representing captured visible lightin some examples and may refer to image data representing captured depthinformation and/or thermal information in other examples.

“Processing image data” or variants thereof herein refers to one or morecomputer-implemented functions performed on image data. For example,processing image data may include, but is not limited to, imageprocessing operations, analyzing, managing, compressing, encoding,storing, transmitting and/or playing back the video data. Analyzing theimage data may include segmenting areas of image frames and detectingobjects, tracking and/or classifying objects located within the capturedscene represented by the image data. The processing of the image datamay cause modified image data to be produced, such as compressed and/orre-encoded image data. The processing of the image data may also causeadditional information regarding the image data or objects capturedwithin the images to be outputted. For example, such additionalinformation is commonly understood as metadata. The metadata may also beused for further processing of the image data, such as drawing boundingboxes around detected objects in the image frames.

Referring now to FIG. 1, therein illustrated is a block diagram of acamera device 10 according to an example embodiment. The camera device10 is illustrated according its operational modules. An operationalmodule of the camera device 10 may be a hardware component. Anoperational module may also be implemented in hardware, software orcombination of both.

The camera device 10 includes one or more processors, one or more memorydevices coupled to the processors and one or more network interfaces.The memory device can include a local memory (e.g. a random accessmemory and a cache memory) employed during execution of programinstructions. The processor executes computer program instruction (e.g.,an operating system and/or application programs), which can be stored inthe memory device.

In various embodiments the processor may be implemented by anyprocessing circuit having one or more circuit units, including a digitalsignal processor (DSP), graphics processing unit (GPU) embeddedprocessor, etc., and any combination thereof operating independently orin parallel, including possibly operating redundantly. Such processingcircuit may be implemented by one or more integrated circuits (IC),including being implemented by a monolithic integrated circuit (MIC), anApplication Specific Integrated Circuit (ASIC), a Field ProgrammableGate Array (FPGA), etc. or any combination thereof. Additionally oralternatively, such processing circuit may be implemented as aprogrammable logic controller (PLC), for example. The processor mayinclude circuitry for storing memory, such as digital data, and may, forexample, include the memory circuit or be in wired communication withthe memory circuit.

In various example embodiments, the memory device is communicativelycoupled to the processor circuit and is operable to store data andcomputer program instructions. Typically, the memory device is all orpart of a digital electronic integrated circuit or formed from aplurality of digital electronic integrated circuits. The memory devicemay be implemented as Read-Only Memory (ROM), Programmable Read-OnlyMemory (PROM), Erasable Programmable Read-Only Memory (EPROM),Electrically Erasable Programmable Read-Only Memory (EEPROM), flashmemory, one or more flash drives, universal serial bus (USB) connectedmemory units, magnetic storage, optical storage, magneto-opticalstorage, etc. or any combination thereof, for example. The memory devicemay be operable to store memory as volatile memory, non-volatile memory,dynamic memory, etc, or any combination thereof.

In various example embodiments, a plurality of the components of theimage capture device may be implemented together within a system on achip (SOC). For example, the processor, the memory device and thenetwork interface may be implemented within a SOC. Furthermore, whenimplemented in this way, both a general purpose processor and DSP may beimplemented together within the SOC.

The camera device 10 includes at least one camera module 16 (forconvenience of illustration only one is shown in the illustrated exampleembodiment) that is operable to capture a plurality of images andproduce image data representing the plurality of captured images. Thecamera module 16 generally refers to the combination of hardware andsoftware sub-modules that operate together to capture the plurality ofimages of a scene. Such sub-modules may include an optical unit (e.g.camera lens) and an image sensor. In the case of a digital cameramodule, the image sensor may be a CMOS, NMOS, or CCD type image sensor.

The lens and sensor combination defines a field of view. When positionedat a given location and at a given orientation, the camera module 16 isoperable to capture the real-life scene falling within the field of viewof the camera and to generate image data of the captured scene.

The camera module 16 may perform some processing of captured raw imagedata, such as compressing or encoding the raw image data.

The camera device 10 may optionally include a video analytics module 24.The video analytics module 24 receives image data from the camera module16 and analyzes the image data to determine properties orcharacteristics of the captured image or video and/or of objects foundin scene represented by the image or video. Based on the determinationsmade, the video analytics module 24 may further output metadataproviding information about the determinations. Examples ofdeterminations made by the video analytics module 24 may include one ormore of foreground/background segmentation, object detection, objecttracking, object classification, virtual tripwire, anomaly detection,facial detection, facial recognition, license plate recognition,identifying objects “left behind”, monitoring objects (i.e. to protectfrom stealing), and business intelligence. However, it will beunderstood that other video analytics functions known in the art mayalso be implemented by the video analytics module 24.

The camera device 10 may optionally include a video management module32. The video management module 32 receives image data and performsprocessing functions on the image data related to video transmission,playback and/or storage. For example, the video management module 32 canprocess the image data to permit transmission of the image dataaccording to bandwidth requirements and/or capacity. The videomanagement module 32 may also process the image data according toplayback capabilities of a client device that will be playing back thevideo, such as processing power and/or resolution of the display of theclient device. The video management 32 may also process the image dataaccording to storage capacity in the camera device 10 or in otherdevices connected to the camera device 10 over a network.

The camera device 10 may optionally include a set 40 of storage modules.For example, and as illustrated, the set 40 of storage modules include avideo storage module 48 and a metadata storage module 56. The videostorage module 48 stores image data, which may be image data processedby the video management module 32. The metadata storage module 56 storesinformation data output from the video analytics module 24.

It will be understood that while video storage module 48 and metadatastorage module 56 are illustrated as separate modules, they may beimplemented within a same hardware storage device whereby logical rulesare implemented to separate stored video from stored metadata. In otherexample embodiments, the video storage module 48 and/or the metadatastorage module 56 may be implemented within a plurality of hardwarestorage devices in which a distributed storage scheme may beimplemented.

The storage modules 48, 56 provide non-transitory storage of image dataand/or metadata. In other example embodiments wherein storage modules48, 56 are not provided, image data generated by the camera module 16and metadata generated by the video analytics module 24 may beimmediately transmitted to an external device over a network.

The camera device 10 includes a networking module 64 operable forproviding data communication with another device over a network 72. Thenetwork 72 may be a local area network, an external network (e.g. WAN,Internet) or a combination thereof. In other examples, the network 72may include a cloud network.

The camera device 10 further includes a transceiver 80 operable forcommunicating wirelessly with another device. The wireless communicationmay be provided according to any protocol known in the art, such asBluetooth, Wi-Fi, ZigBee or cellular communication. Alternatively,camera device 10 may be operable for communicating over a wire withanother device.

In some examples, the transceiver 80 is a short-range, low-powertransceiver. A short-range, low-power transceiver may be useful forreducing power consumption of the external device with which the cameradevice 10 is communicating. For example, the transceiver 80 that isshort-range may have a communication range of less than about 10 meters.For example, the transceiver 80 that is low-power may have a powerconsumption of less than about 0.5 Watts. A short-range, low-powertransceiver may implement a low-energy Bluetooth or low-energy Wi-Fiprotocol known in the art.

The camera device 10 further includes a wireless power transmitter 88that is operable for transmitting power wirelessly to an externalelectrical load. The external electrical load may be an energy storagedevice, such as at least one battery or capacitor. For example, powermay be transferred by magnetic fields in which one or more coils ofwires in the wireless power transmitter 88 is coupled by magneticinduction with a cooperating coil in the external device that is beingpowered by the wireless power. The inductive coupling between thewireless power transmitter 88 and a cooperating device receiving thepower may be resonant inductive coupling or electrodynamic induction. Itwill be understood that the wireless power transmission is not limitedto non-radiative techniques. In some examples, longer range techniquesmay be used, such as power beaming based on microwaves or lasers.

The camera device 10 further includes a power supply 96 operable forsupplying electrical power to the hardware components of the cameradevice 10, such as those implementing the camera module 16, transceiver80, networking module 64 and wireless power transmitter 88.

In some examples, the power supply 96 receives electrical power from apower source over a wired connection. The power source may be mainselectricity (ex: 110V/220V AC), which may be converted to a supplysuitable for the camera device 10 (ex: converting to DC, rectifying to alower voltage). In some alternative examples, the power source may be anintermediate device that supplies power in addition to performinganother function, such as processing or networking. In yet furtheralternative examples, the power supply may be supplying power in asustainable manner based on, for instance, solar power technology orpower received wirelessly from another device in communication with thecamera device 10.

In one example embodiment, power may be supplied to the power supply 96over a connection that is also providing data communication. Forexample, power may be supplied to the power supply 96 by power overEthernet (POE), wherein the cable connected to the networking module 64for network data communication is also used for supplying power to thepower supply. As illustrated, the same cable 104 that is connected tothe network (e.g. connected to a network switch or router) is alsoconnected to the power supply 96.

The camera device 10 may further include a power management module 112that is operable for managing the supply of power from the power supply96 to various hardware components of the camera device 10. The powermanagement module 112 may manage the power being consumed by thewireless power transmitter 88 separately from management of power beingconsumed by other components of the camera device 10. The powermanagement module 112 may further control the priority of providingpower to various modules of the camera device 10. This prioritization inthe case of high power demand from various modules, which may otherwisecause system overload.

For example, a wireless power transmitter power management submodule maycontrol the power level of the wireless power transmitted from thewireless power transmitter 88. The power level may be varied accordingto characteristics of an external device receiving the wireless power.Such characteristics may include one or more of the distance of theexternal device from the camera device 10, the average power requirementof the external device, the instantaneous power requirement of theexternal device, and the current battery status of the external device.

The power level may also be varied according to environmental factors,such as time of day, location, and number of proximately locateddevices. For example, where the camera device 10 is used for chargingthe external device, the wireless power transmitter power managementsubmodule may choose to transmit wireless power for charging duringoff-peak hours.

The power level may also be varied according to power load requirementsfrom other components of the camera device 10. For example, duringperiods when other components of the camera device 10 experience heavyload, the power management module 112 may supply less or no power to thewireless power transmitter. These periods may occur when the cameradevice 10 has to handle a large amounts of data, such as transferring orbacking up data stored within the storage module 40.

The example camera device 10 is suitable for use in conjunction with anexternal device that requires data communication with another deviceover a network and that would benefit from receiving wirelesslytransmitted power. The camera device 10 can provide network connectivityto the external device via data communication provided between thewireless transceiver 80 of the camera device 10 and a correspondingwireless transceiver of the external device. The network connectivity isfurther provided through the connection of the networking module 64 ofthe camera device 10 with the network 72. Accordingly, the externaldevice may be in communication another network node connected to thenetwork 72 only via the camera device 10 and without requiring some formof wired and/or wireless connection from the external device to thenetwork 72.

The camera device 10 can further provide a continued power source forthe external device via wireless power transmitter 88 transmitting powerto the external device. The external device may be battery-operated andthe power transmitted wirelessly from the camera device 10 may be usedto charge at least one battery of the external device. Accordingly, theexternal device may operate without having to receive power over a wiredpower cable. Furthermore, even where the external device may be fullybattery-operated, the providing of wireless power from the camera device10 to the external device to charge the battery of the external devicemay eliminate, or reduce the frequency, of having to change the battery.

In some example embodiments, the power output from the wireless powertransmitter 88 may be variably controlled. For example, the level ofpower output may be adjusted according to the power consumption of theexternal device receiving the wirelessly transmitted power. The level ofpower out may also be adjusted based on one or more parameters of thedeployment of the camera device 10 with the external device, such as thedistance there between. The power output from the wireless powertransmitter 88 may be adjusted so that the level of wireless powerreceived at the external device corresponds with a power requirement ofthe external device, such as an average power requirement of theexternal device. The power output may also be adjusted based on a changein power requirement of the external device. However, the power outputfrom the wireless power transmitter 88 may be throttled by the powermanagement module 112 to ensure continued proper functioning of thecamera device 10. In some example embodiments, the wireless powertransmitter 88 may implement trickle charging or slow charging of theexternal device.

In some example embodiments, the wireless power transmitter 88 may bechosen to provide at least 3 watts power to a power-receiving externaldevice located at a distance of at most 10 meters from the camera device10. For example, such a power output would effectively charge a depthsensor, radar sensor, or typical PIR motion sensor.

In other example embodiments, the wireless power transmitter 88 may bechosen to provide substantially less power, such as about 0.2 mW ofpower at a distance of at most 10 meters from the camera device 10. Thislevel of power output is suitable for external devices that aretypically on standby, such as a smoke alarm.

Referring now to FIG. 2, therein illustrated is a block diagram of acombined system 200 according to one example embodiment having a cameradevice 10 and a sensor system 208.

The sensor system 208 may be a radar sensor, a depth sensor, or both, asdescribed below. Sensor system 208 includes a sensor 216.

The sensor system 208 may include a storage module 224. The storagemodule 224 may be operatively connected with sensor 216 to receivesensed signals and store the sensed signals. The storage module 224 mayalso store one or more sensing rules. The sensor 216 may implementsensing based on applicable sensing rules. For example, the rules maycause the sensor 216 to cease sensing during given periods of the day,for example during daylight, and carry out sensing at other periods ofthe day, for example during the night, when persons are most likely tobe loitering or sleeping in an ATM vestibule.

The sensor system 208 includes a communications transceiver 232 operablefor providing data communication with the camera device 10 via thetransceiver 80. The communications transceiver 232 of the sensor system208 may implement a wireless communication protocol that is compatiblewith the communication protocol implemented by the transceiver 80 of thecamera device 10. For example, the communications transceiver 232 mayalso be a short-range, low-power transceiver.

Sensed signals generated by the sensor 216 can be transmitted fromsensor system 208 using its communications transceiver 232 and receivedat the camera device 10 using its transceiver 80. The sensed data may befurther transmitted to external network device 264 from the cameradevice 10 over the network 72.

The sensor system 208 may further receive commands from the cameradevice 10. The commands may have been initially transmitted from theexternal network device 264 to the camera device 10 via the network 72and the networking module 210 of the camera 10. For example, thecommands may be for controlling the sensor system 208, such as commandsfor changing sensing rules applied to the sensor system 208.

The sensor system 208 further includes a wireless power receiver 240that is operable for receiving power transmitted wirelessly from thewireless power transmitter 88 of the camera device 10. The wirelesspower receiver 240 is configured to be compatible with the wirelesspower transmitter 88 of the camera device 10. For example, the wirelesspower receiver 240 includes one or more coil of wires in which a flow ofelectrical current is induced by the wireless power transmitted from thecamera device 10.

The sensor system 208 may further include at least one battery 248 orother suitable form of power storage device for supplying power to oneor more components of the sensor system 208. The at least one battery248 may supply power to the sensor 216, and the communicationstransceiver 232. The at least one battery 248 is rechargeable usingpower transmitted wirelessly from the camera device 10 and received bythe wireless power receiver 240.

The sensor system 208 may further include a battery management module256. The battery management module 256 operates to manage charging ofthe at least one battery 248 using the power received by the wirelesspower receiver 240.

In one example embodiment, the battery management module 256 may sensethe charge level of the at least one battery 248 and implements chargingof the battery 248 when the charge level falls below a predeterminedlevel.

In another example embodiment, the battery management module 256 mayimplement charging of the battery 248 any time wireless power isavailable from the wireless power receiver 240. The battery managementmodule 256 may be further operable to implement trickle charging or slowcharging of the battery 248.

In yet another example embodiment, the battery management module 256 maybe further operable to sense the battery charge level and to communicatethe charge level to the camera device 10 using the wireless transceiver232. The camera device 10 may be configured to transmit wireless poweronly when it receives an indication that the charge level of the atleast one battery 248 of the radar device 208 has fallen below apredetermined level. Additionally, or alternatively, the batterymanagement module 256 may transmit, using the wireless transceiver 232,a request to the camera device 10 to begin wireless transmission ofpower to the sensing device so that the power can be used for chargingthe at least one battery.

Continuing with FIG. 2, the camera device 10 is operable to transmitover the network 72 sensed signals received from the sensor system 208.Accordingly, the camera device 10 acts as a gateway to the network 72for the sensor system 208. The camera device 10 may transmit the imagedata and the sensed data to their respective destinations over thenetwork 72.

In various example embodiments, the camera device 10 may be configuredto transmit the sensed signals received from the sensor system 208 tothe same destination networked device 264 over the network 72. Forexample, the destination networked device 264 may be a server thatprocesses or manages the image data and/or the sensed signals. Whenbeing transmitted to the same destination networked device, image datathat is captured by the camera module 16 at a given time is logicallyassociated with sensed signals pertaining to one or more conditionssensed by the sensor 216 at the same time. “Logically associated” hereinrefers to an association in which knowledge of the relevant image dataallows retrieval of its logically associated sensed signals and viceversa. For example, the image data and its corresponding signal may bothinclude a time stamp, which provides the logical association.

According to various example embodiments wherein the camera device 10 isused in a video surveillance application to visually monitor an area orasset, such as an ATM vestibule, the condition sensed by the sensorsystem 208 may provide information about the area or asset, which mayprovide enhanced monitoring. For example, the radar signals or depthcamera images may be used to confirm or provide further informationregarding an event that is captured by the camera device 10. Thisinformation may be also be used to confirm or improve certainty of adetermination made by the video analytics module 24.

In some example embodiments, the video analytics module 24 may determineproperties or characteristics of the captured image or video and/or ofobjects found in the scene represented by the image or video based on acombination of analysis of the image data and one or more relevantsignals from sensor system 208. Relevant signals sensed by the sensorsystem 208 may be conditions sensed during a time period correspondingto the time period of the image data being analyzed.

According to various example applications, the sensor system 208 islocated in proximity of the camera device 10, such as within the samephysical area. For example, the sensor system 208 may be located suchthat signals received by the sensor system 208 are relevant to the imagedata captured by the camera device 10. Accordingly, the signals receivedmay serve to enhance the monitoring performed using the camera device10. It will be appreciated that the proximity of the camera device 10with the sensor system 208 allows for effective wireless transmission ofpower from camera device 10 to the sensor system 208 and for effectivewireless data communication between the camera device 10 and the sensorsystem 208. This allows the sensor system 208 to operate fullywirelessly (i.e. without requiring a wired connection for datacommunication with an external device and for receiving power). It willbe further appreciated that even in other examples where the sensorsystem 208 generates signals that is not pertinent to the image datacaptured by the camera device 10, the interaction between the cameradevice 10 and the sensing device 10 allows the sensor system 208 tooperate fully wirelessly.

Referring now to FIG. 4A, therein illustrated is a block diagram of acamera device 10′ according to an alternative example embodiment. Thealternative camera device 10′ includes the same operational module asthe camera device 10 illustrated in FIG. 1 and the description providedherein regarding these modules are also applicable to the alternativecamera device 10′. The alternative camera device 10′ is different inthat it includes a plurality of wireless power transmitters 88. In theillustrated example, the alternative camera device 10′ includes nwireless power transmitters 88. Power supplied to each of the pluralityof power transmitters 88 may be controlled by the power managementmodule 112.

The plurality of power transmitters may each be used to power differentsets of one or more sensor systems 208 and/or access control devices308. For example, the sensor systems 208 may be sparsely located suchthat a single power transmitter 88 cannot effectively provide power toall of the sensor systems 208. The wireless transceiver 80 of thealternative camera device 10′ may be used for simultaneous datacommunication with a plurality of sensor systems 208 to which thealternative camera device 10′ is transmitting power. For example, anappropriate multiplexing scheme may be used to maintain datacommunication with each of the plurality of sensor systems 208.

In one example embodiment, at least one of the wireless powertransmitters 88 of the alternative camera device 10′ is pivotable so asto change an orientation of the wireless power transmitters 88. Forexample, a wireless power transmitter 88 may be angularly pivoted to besufficiently aligned with an external sensor system 208 so as toeffectively transmit power to that external device.

Referring now to FIG. 4B, there illustrated is a schematic diagram of anexample deployment 380′ of an alternative camera device 10′ according toone example embodiment. An example alternative camera device 10′ havinga circular form factor is provided. The example alternative cameradevice 10′ is a multi-transmitter camera and includes three wirelesspower transmitters 88. Three sensor systems 208 are located around thecamera device 10′. Each of the wireless power transmitters 88 arealigned with the location of a sensor system 208 so as to providewireless power to that sensor system 208.

Referring now to FIG. 5, therein illustrated is a block diagram ofconnected devices of a video surveillance system 400 according to oneexample embodiment.

The system 400, as depicted, includes at least one camera device. In theexample illustrated in FIG. 5, a first camera device 10 a, a secondcamera device 10 b, and a third camera device 10 c are each connected tothe network 72. Image data and/or metadata generated by the cameradevices 10 a, 10 b, and 10 c are transmitted to other network-connecteddevices over the network 72.

In the illustrated example, the third camera device 10 c is connected tothe network 72 through a processing appliance 404. The processingappliance 404 is operable to process the image data output by the thirdcamera device 10 c. The processing appliance 404 includes one or moreprocessors and one or more memory devices coupled to the processor. Theprocessing appliance 404 may also include one or more networkinterfaces.

The first camera device 10 a is in data communication with a firstsensor system 208 a using their respective wireless transceivers 80,232. Signals and data generated by the sensor system 208 a istransmitted over the network 72 via the first camera device 10 a. Thefirst camera device 10 a further transmits power wirelessly from itswireless power transmitter 88. The transmitted power is received at thesensing device 240 of the first sensor system 208 a, which may be usedto charge its one or more batteries or energy storage devices.

The second camera device 10 b is in data communication with a secondsensor system 208 b. It will be understood that although two antennasare illustrated, a single wireless transceiver 80 in the second cameradevice 10 b may be in data communication with the second sensor system208 b. Sensed signals and data generated by the second sensor system 208b is transmitted over the network 72 via the second camera device 10 b.

In another example embodiment, the second camera device 10 b ismulti-transmitter device, as described herein with reference to FIG. 4A,and includes a first wireless power transmitter 88 a and a secondwireless power transmitter 88 b.

The third camera device 10 c is not transmitting wireless power. Thethird camera device 10 c may be a standard IP camera that does not havewireless power transmission capabilities.

The system 400 may further include a third sensing device 208 c, whichmay be an environmental sensing device, which is in direct connectionwith the network 72.

The system 400 includes at least one workstation 408 (e.g. server), eachhaving one or more processors. The at least one workstation 408 may alsoinclude storage memory. The workstation 408 receives image data from atleast one camera device 10 and performs processing of the image data.The workstation 408 may further send commands for managing and/orcontrolling one or more of the camera devices 10. The workstation 408may receive raw image data from the camera device 10. Alternatively, oradditionally, the workstation 408 may receive image data that hasalready undergone some intermediate processing, such as processing atthe camera device 10 and/or at a processing appliance 404. Theworkstation 408 may also receive metadata from the image data andperform further processing of the image data.

The video capture and playback system 400 further includes at least oneclient device 164 connected to the network 72. The client device 164 isused by one or more users to interact with the system 400. Accordingly,the client device 164 includes at least one display device and at leastone user input device (for example, mouse, keyboard, touchscreen, joystick, microphone, gesture recognition device, etc.). The client device164 is operable to display on its display device a user interface fordisplaying information, receiving user input, and playing back imagesand/or video. For example, the client device may be any one of apersonal computer, laptops, tablet, personal data assistant (PDA), cellphone, smart phone, gaming device, and other mobile and/or wearabledevices.

Referring back to FIG. 2, the wireless power transmitter 88 transmitswireless power over an effective powered space. The effective poweredspace refers to the space in which a wireless power receiver 240 may belocated and effectively receive the wirelessly transmitted power. Awireless power receiver 240 may be considered to be effectivelyreceiving wireless power if the power at the receiver 240 exceeds apredetermined power threshold. Alternatively, a wireless power receiver240 may be considered to be effectively receiving wireless power if thepower at the receiver 240 induces a current in the receiver 240 thatexceeds a predetermined current threshold.

According to various example embodiments, the field of view of thecamera device 10 substantially overlaps with the effectively poweredspace of the wireless power transmitter 88. The field of view of thecamera device 10 may be fully encompassed within the effectively poweredspace of wireless power transmitter 88. The field of view of the cameradevice 10 may be fully encompassed in that the effectively powered spaceoccupies a larger space than the field of view. However, it will beunderstood that the field of view of the camera device 10 may extendpast the outer limit of the effectively powered space based on adistance from the camera device 10.

By ensuring that the field of view of the camera device 10 is fullyencompassed within the effectively powered space of the wireless powertransmitter 88, any object that falls within the field of view will alsobe within effectively powered space of the wireless power transmitter 88and can receive wireless power therefrom (so long as the distance of theobject does not exceed the outer limit of the operational space). Thismay facilitate installation of a sensor system 208 in that the installeronly needs to place the sensor system 208 within the field of view ofthe camera device 10 to ensure that the sensor system 208 will beproperly receiving wireless power from the camera device.

According to various example embodiments wherein the optical unit of thecamera device 10 is pivotable to change the field of view of the cameradevice 10, the wireless power transmitter 88 is configured to maintainthe overlap of the field of view with the operational space. Thewireless power transmitter 88 may be configured to maintain the field ofview being fully encompassed with the effectively powered space over therange of pivotable motion of the optical unit of the camera. Examples ofcameras with a pivotable optical unit include a dome camera and apan-tilt-zoom (PTZ) camera.

In one example embodiment, the wireless power transmitter 88 transmitspower directionally. Accordingly, the operational space of the wirelesspower transmitter is defined by an effective power coverage cone. Thewireless power transmitter 88 and the optical unit of the camera module16 may be substantially aligned so that the field of view of the cameradevice 10 overlaps with the power coverage cone. The alignment may besuch that the field of view of the camera device is fully encompassedwithin the power coverage cone.

Referring now to FIG. 6, therein illustrated is a schematic diagram ofan example deployment of a camera device 10 and sensor system 208 and anAutomatic Teller Machine (ATM) 530. The camera device 10 has a field ofview 508, which may be substantially conical. In the illustratedexample, the field of view 508 is defined by its upper boundary 512 andlower boundary 516. The sensor system 208 has a field of view 520defined by its upper boundary 524, lower boundary 528 and outer limit532. It will be appreciated that the field of view 508 is fullyencompassed within field of view 520 (but for a space close to theoptical unit of the camera device 10 and a space outside of the outerlimit 532). The camera device 10 and sensor system 208 is oriented so asto capture ATM 530 or an area around ATM 530, such as an ATM vestibule,which is the protected asset.

Radar Sensor

Referring now to FIG. 3A, sensor system 208 as described above, mayinclude a radar system 300. Radar system 300 may be powered wirelessly,as described above, or via alternative means, including a direct powerconnection to a power source, such as using Power over Ethernet (POE).The radar system 300 may include two radar devices 302A and 302B, eachcommunicatively coupled to camera device 10, for example using a cableconnected to relay contacts; and power adaptor 304, for example using apower cable, including for example a 5 VDC and a ground cable. Poweradaptor 304 converts signals received from POE switch 308, for examplefrom an Ethernet cable, into power for radar devices 302A, 302B, andcamera device 10. Data signals are sent from radar devices 302A, 302B tocamera device 10 for further processing at camera device 10, or sent bycamera device 10 through POE switch 308, using for example an Ethernetcable, for further processing. It is appreciated that while theembodiment shown in FIG. 3A does not employ a wireless power system, itmay be adapted to use such a wireless power system as described above.In alternative embodiments, radar system 300 may include an accesscontrol panel or alarm intrusion panel in place of, or in addition to,camera device 10. Alternatively, radar device 302A may be a standalonedevice providing alerts directly to network 72.

Referring now to FIG. 3B, video surveillance and radar system 320include radar systems 302A, 302B, 302C, and 302D, each communicativelycoupled, for example via a USB connection, to a radar system hub 350 toprovide data to and receive instructions from radar system hub 350, andreceive power through radar system hub 350. Radar system hub 350 iscommunicatively coupled to both POE adaptor 304, for example through a 5VDC and ground line, to receive power through POE adaptor 304; and tocamera device 10, to which radar system hub 350 sends data signals forfurther processing. POE adaptor 304 and camera device 10 are eachcommunicatively coupled, for example via Ethernet cables, to POE switch308.

Radar system hub 350 may be mounted above the ceiling of the locationbeing monitored, and includes a processor 360 to configure and collectdata received from radar systems 302A, 302B, 302C, and 302D. If apresence is detected by one of radar systems 302A, 302B, 302C, and 302Da relay system 370 in radar system hub 350 is energized and relays amessage to camera device 10 and/or the network 72.

Referring now to FIG. 7, therein illustrated is a schematic diagram ofan example ceiling deployment of a camera device 10 and radar system300. Camera device 10, which is shown as a fisheye camera, but may be adome camera or PTZ camera, with field of view 704, may be mounted inenclosure 720. Enclosure 720 is secured to ceiling 710 of, for example,an ATM vestibule. Radar system 300, with field of view 708, may bepositioned in enclosure 720 adjacent to camera device 10, so that fieldof views 704 and 708 overlap. In this embodiment camera device 10 canprovide additional data related to alerts relayed by radar system 300.

Referring now to FIG. 8, therein illustrated is a block diagram of anexample embodiment of a camera device 10 and radar system 300 within ahousing 804. Radar system 300 may be communicatively coupled, via acable, such as a USB cable, to camera device 10 within housing 804.Camera device 10 may receive power from and output data to POE switch308 through a cable, such as an Ethernet cable.

Referring now to FIG. 9, therein illustrated is a block diagram of anexample embodiment of a radar system 300. Radar system 300 includesprocessor 902, which may be an ARM-based CPU or similar CPU, and whichreceives power, which may be received wirelessly, via POE, or othermeans. Processor 902 receives input from radar transceiver 906, whichmay be an Ultra-Wideband (UWB) transceiver and outputs to camera device10 through relay 904. Controller 914, communicatively coupled toprocessor 902 and which may be a Breakout board, controls indicators,such as LEDs 910 and may be operated by switches 912.

Referring now to FIG. 10, therein illustrated is an embodiment of anexample of a radar system 300. Radar system 300 includes enclosure 1002,to protect the internal elements of radar system 300. Enclosure 1002 ismade of material transparent to radar signals. Opposite enclosure isback plate 1004, typically a flat plate to meet with a surface formounting radar system 300. Aperture 1008 allows a cable or otherconnector to enter enclosure 1002. LEDs 910 positioned on enclosure 1002can be configured to provide status information regarding radar device208.

Referring now to FIGS. 11A and 11B, therein illustrated are detectionareas provided by various example placements of radar system 300. Asshown in FIG. 11A, radar system 300, when mounted to wall 1105 may bepositioned to provide a detection area defined by a semicircle 1110bisected by the meeting between wall 1105 and floor 1115. As shown inFIG. 11B, when radar system 300 is mounted to the ceiling 1120, acircular detection area 1125 can be provided.

Radar system 300 works by transceiver 906 sending and receiving radarsignals. The returning signal will indicate the distance to a detectedobjected and the Doppler Effect is used to determine a portion of thevelocity of the detected object as indicated by the change in frequencyof the returned radar signal as determined using a Fouriertransformation. Comparing signals over time allows processor 902 todetermine the direction of the detected object's motion.

Radar system 300 may be used for a number of purposes, includingidentifying the presence of a person in a location, such as a dressingroom, a prison cell, or ATM vestibule, by detecting biometric indicatorssuch as breathing or heartbeats. Detection of a human being as a livingobject, and not as a motionless object, can be performed by short-rangeradars using microwave signals ranging in frequency, waveform, duration,and bandwidth. Radar system 300 can detect people not actively moving,only breathing and with a heartbeat, and thereby determine the presenceof a sleeping person.

On reflection from a person, a radar signal acquires specificbiometrical modulation, which does not occur in reflections frominanimate objects. This modulation is produced by heartbeats, pulsationsof vessels, lungs, and skin vibrations in the region of the person'sthorax and larynx, which occur synchronously with breathing motions andheartbeat. These processes are nearly periodic, with typical frequenciesin the range of 0.8^(−2.5) Hz for heartbeat and 0.2^(−0.5) Hz forbreathing. Therefore, the delay or phase of the reflected signal isperiodically modulated by these periodic oscillations. The modulationparameters are thus determined by the frequencies and intensities ofrespiration and heartbeat.

The sensitivity of radar probing in the gigahertz band may reach 10⁻⁹ m.In practice, radar probing of live persons is performed against thebackground of reflections from local objects; as a rule, the intensityof these reflections exceeds the intensity of signals from a humanobject. Human objects, however, are distinguished by periodic andaperiodic modulation synchronous with the respiration and heartbeat of aperson. Modulation of this type is either absent in signals reflectedfrom local objects or has different time and spectral characteristics.This allows for recognition of signals reflected by a human personagainst the background reflections from local objects.

Radar systems 300 may use probing signals of different types, forexample unmodulated monochromatic signals, UWB video pulses, andwideband SFM signals. The main advantage of wideband and UWB signalsover monochromatic signals is that they allow the range separation oftargets from exterior interference, such as reflections from localobjects.

Radar Use Case

Referring now to FIG. 21, radar system 300 may be used with cameradevice 10 to detect the presence of individuals in an area such as anATM vestibule. ATM vestibules are often open to the public and can beaccessed by anyone. Radar system 300 sends pulses out and detects thepresence of a person by determining if the returned radar signalsrepresent biometric signals (step 1900). If not, radar system 300resumes scanning, and if the presence of a person is detected a timerbegins (step 1910) and is incremented (step 1920). If the person isstill detected (step 1925) and a predetermined period of time haspassed, for example ten minutes (step 1930), relay 904 alerts cameradevice 10 (step 1940), or alternatively network 72. If camera device 10has already determined the presence of a person (step 1950), for exampleby using analytics module 24, then no further action need be taken.However, if camera device 10 is not yet alerted to the presence of aperson, then the alert is escalated (step 1960), and may, for exampleresult in: an alert to a security guard to review images from cameradevice 10, or for the security guard to attend to the ATM vestibule.

The biometric signals received can also be used to detect if the personis asleep or not, or is undergoing a health emergency (for example hasan erratic heartbeat, which if detected could be used to alert emergencypersonnel), and can be used to detect persons not otherwise moving.

In an embodiment, radar system 300 or a depth sensor 208 (as describedbelow) when used in an ATM vestibule to detect loitering, may use a doorstatus switch to reset the timer. Each time the door opens, for examplewhen a new customer enters, the timer may be reset so that during thehigh traffic periods there is no alarm. Typically, people are loiteringor sleeping in low traffic periods when the door does not usually openand close frequently.

Alternatively, instead of an ATM vestibule, radar system 300 could beused to detect the presence of a person in an ATM back room or a bankvault.

Depth Sensor

A depth map (or depth image) is an image that includes informationrelating to the distance of the surfaces of scene objects from aviewpoint such as from a depth sensor such as a 3D camera. For eachpixel, or group of pixels, in the image of the depth map; there isassociated a distance from the depth sensor. Depth maps can use a numberof different means to show distance such as by luminance in proportionto the distance to the depth sensor, and by color. An example ofluminance in proportion to the distance may be further distances darkerand nearer distances lighter in a gray scale image, alternatively, itmay be further distances lighter and nearer distances darker. An exampleof color depth map may use the red green blue (RGB) spectrum: red forfurther distances, yellow/green for middle distances, and blue forcloser distances.

Depth sensors may use a number of different technologies to create depthmaps. The technologies include Time-of-Flight (ToF), Stereo, andStructured Light.

Referring to FIG. 12, there is shown an embodiment of an exampleinstallation of two 3D cameras 1202 on the ceiling of a room 1206. The3D cameras being structured light 3D Cameras which provide both 2Dimages and depth maps (or depth images). A computer 1204 to process theimages of the two 3D cameras 1202 is also shown. The room 1206 could bea vestibule to ATMs (automatic teller machines) or be a buildingentrance. The room 1206 could include any area or zone undersurveillance whether inside a building or outside of a building.

As shown in FIG. 12, the two 3D cameras 1202 are in an overhead modewhich has the best chance of getting an approximate ‘size’ of theobject. However, the overhead mode cameras cannot see what is not in thedirect line of sight, for example: a square box is continuous from thetop surface of the box all the way to the floor, however, a pyramid canalso have an approximate volume (assuming the base is flat against thefloor). If, however, you balance the pyramid on the point with the flatpart facing the camera, then it will appear as a box to the 3D cameras.For a ball resting on the floor, only the top hemisphere is visible bythe camera so the volume calculated would not be for a sphere butinstead for a box for the bottom half of the diameter and a hemispherefor the top half. This is a limitation of line of sight range (distance)finding depth sensors such as the two 3D cameras 1202.

For a number of applications, getting an approximate ‘size’ (or rough‘volume’) of an object is sufficient. It may also be sufficient to justcount the number of pixels above a certain height threshold which iscloser to calculating the surface area of the object. Of course, onceyou have the surface area and the depth or height, the volume is easilycalculated. For certain applications, only counting surfaces above acertain threshold is used to filter out “thin” objects that may be onthe floor like a paper cup or piece of paper. Any “thick” objects above,for example, 4 inches is of interest.

The 3D cameras 1202 are angled slightly for a larger field of view; the‘thickness’ difference calculation is still sufficient although the sideof the furthest part of the object would now get occluded by the part ofthe object closer to the 3D cameras 1202. In the case of the pyramidwith the base flat on the floor it would now appear to be elongated awayfrom the 3D cameras 1202. If the angle is too steep, then the error maynot be tolerable (say 90 degrees facing one of the side of the pyramid;the pyramid would appear to have the shape of a triangular prism). Thus,the 3D cameras 1202 using a relatively narrow field of view (or smallviewing angles from the axis of the camera view) and not too steeplyangled may achieve more reliable results.

Referring to FIG. 13, there is shown an example mounting for two 3Dcameras 1302, 1304 for a larger field of view. Referring to FIG. 14,there is shown an alternative example mounting for two 3D cameras 1402,1404 for a smaller field of view. While the embodiment of FIG. 12 usestwo 3D cameras 1202, another embodiment may use only one 3D camera andthe disclosure as shown in FIGS. 15 to 19 as described below would stillapply accordingly. A further embodiment may use more than two 3D camerasand the disclosure as shown in FIGS. 15 to 19 would still applyaccordingly.

Referring to FIG. 15, there is shown example images from theinstallation of FIG. 12. Shown is a 2D image 1502 and its correspondingdepth map 1506. As an example, a person is shown standing in the 2Dimage 1502 and in the corresponding depth map 1506. The depth map 1506is displayed using a color map (RGB spectrum) to better visualize thedepth information (and shown in grayscale in FIG. 15). Depth map 1506without the person and a depth map without the person are together thebackground or the model of the background; the background being theinstallation room with its floors, walls and any other stationaryobjects. The model of the background, for example, is composed ofaverage depths from 1000 frames (or camera shots) of the depth maps1506, and the depth map without a person (when the area undersurveillance has no objects in the field of view of the depth sensor)for each of the pixels or group of pixels. Alternatively, the model ofthe background, for example, is composed of least distances to the 3Dcameras 1202, 1204 from 1000 frames of the depth maps 1506, 1508 foreach of the pixels or group of pixels.

Referring to FIG. 16, there is shown additional example images from theinstallation of FIG. 12. There is a two 2D image 1602 and itscorresponding delta depth map 1606. There are no objects or people shownin the 2D image and the corresponding delta depth map 1606. The deltadepth map 1606 is the net difference between subtracting (or comparing)the depth maps (generated corresponding to the 2D image 1602) from themodel of the background. The delta depth map 1606 represents thedisplacement of an object or objects from the floor of the installation,and would be the foreground. Due to noise, the delta depth map 1606 maynot always represent zero displacement, however, within a certain range,for example 1 inch, they are equivalent to zero and is represented asblue in the delta depth map 1606. Further, by setting a threshold of,for example, 4 inches from the floor, “thin” objects, like a paper cupor piece of paper, may be filtered out.

Referring to FIG. 17, there is shown additional example images from theinstallation of FIG. 12 with a person 1710. There is a 2D image 1702 andits corresponding delta depth map 1706. The delta depth map 1706 showsthe person 1710 and is detected by the video analytics module 24 as alarge object. A volume may be calculated from the amount of thedisplacement of a blob (the person 1710) in the delta depth map 1706.Either the volume or the amount of the displacement may then be used toindicate whether it could be a person by the video analytics module 24.

Referring to FIG. 18, there is shown additional example images from theinstallation of FIG. 12 with a person 1810. There is a 2D image 1802 andits corresponding delta depth map 1806. The delta depth map 1806 showsthe person 1810 (the blob 1810) and is detected by the video analyticsmodule 24 as a large object due to the amount of displacement (orvolume). However, since the least depth of the person 1810 in the deltadepth map 1806 is not high and since the volume is sufficient toindicate a person, the video analytics module 24 indicates that theperson 1810 is on the floor. If the blob 1810 remains within the fieldsof view of the 3D cameras 1202 beyond a period of time, then the person1810 is also labelled as sleeping on the floor, or loitering. The periodof time may set by a user depending on the experiences of operating atany particular location of what may be loitering at that particularlocation.

Referring to FIG. 19, there is shown a flowchart of an example of anembodiment of image processing of the installation of FIG. 12. The two3D cameras 1202 capture depth data to create depth maps 1902 which areprocessed to create a model of the background 1904. The model of thebackground 1904 is created by capturing a series (or frames) of depthmaps 1902 and every pixel is updated with the lowest non-zero heightvalue (depth) within a certain time period. Within the certain timeperiod, for example, there is 1,000 frames of the depth maps 1902.

There may be certain limitation with the 3D cameras 1202. The structuredlight 3D Cameras uses infrared (IR) light patterns to detect depth ordistance to target. However, certain types of surfaces (reflectivesurfaces) reflect away the IR patterns of the structured light of 3Dcameras, resulting in no reading (or zero depth) in the depth map.Further, when the ambient IR is strong, the IR patterns can be washedout, resulting in no readings as well. In all cases, in order togenerate a stable and valid background model, the depth value of those“no reading” areas have to be estimated. The estimation is based on theneighbor pixels and is called interpolation. There are various methodsof interpolation that could be used, for example, morphologicalfiltering and bilinear filtering.

The generation of the model of the background 1904 also includesinterpolating the height values (depth) for reflective regions where the3D cameras 1202 is unable to detect the depth. The model of thebackground 1904 may be recalculated periodically. Once calculated, anynew frames of the depth maps 1902 are subtracted from the model of thebackground 1904 to produce corresponding foreground frames 1906 (deltadepth maps). The value of each pixel of the model of the background 1904is subtracted from the value of each corresponding pixel of each frameof the depth maps 1902 to produce the foreground frames 1906 or deltadepth maps. Where there is only one 3D camera, each depth map frame (a3D camera shot) is compared to the model of the background to generate acorresponding foreground frame. The video analytics module 24 thenanalyzes the foreground frames 1906 to detect objects, large objects,and people loitering. The person 1810 is accordingly detected to beloitering after the person 1810 is detected to be in the foregroundframes 1906 for a certain period of time and to be also sleeping on thefloor if the maximum height is below a certain level, for example, 3feet above the floor. The results are then displayed 1908 as shown inFIG. 18 as person on floor 1812.

Referring to FIG. 20, there is shown an example embodiment installationof a radar sensor 2002 and a depth sensor 2004 on the ceiling of a room2006. The radar sensor 2002 may be, for example, a UWB radar sensor. Thedepth sensor 2004 may be, for example, a structured light 3D camera. Thecombination of having two different sensors enhances the reliability ofdetecting objects and any consequential loitering. Each sensor by itselfhas certain weaknesses, for example, a structured light 3D Camera maynot function properly in strong sunlight due to its use of Infraredlight, and the radar sensor 2004 may not be configured to easily detectmore than one object or person. The video analytics module 24 may, forexample, be set to use the outputs from both sensors 2002, 2004 todetect a person or a person loitering, but the video analytics module 24will indicate such detection if only one of the sensors 2002, 2004indicates a person, or a person is loitering. It is also more reliablewhen the outputs of both sensors indicate a person, or a person that isloitering.

Referring to FIG. 6, when the sensor 208 is a depth sensor like astructured light 3D Camera, the generated depth maps may be veryaccurate, for example, having a resolution of up to 1 mm. The depth mapsof the sensor 208 are used to generate a model of the background of theATM 530. The model of the background may be generated as shown in FIGS.18 and 19. The model of the background is then subtracted (or comparedto) from new frames of depth maps taken by the sensor 208 to generateforeground frames. Where the foreground frames show a difference fromthe model of the background then an alert may be generated to indicatethat the ATM 530 may have been tampered by the installation of a façadeto skim the banking data of users. The ATM 530 includes any machinewhere users may enter banking data such as pass code or PIN (personalidentification number) such as gas station pumps and government servicekiosks.

In an alternative embodiment sensor 208, such as radar sensor 300 may bein a portable unit that can be carried by a single user. In thisembodiment, sensor 208 may be powered by a battery or the like, and mayinclude a display to notify users of the detection of a heartbeat orother indicates of a presence.

Alternatively, instead of ATM vestibules, sensor 208, when a radarsystem 300 or a depth sensor, may be used to determine loitering inother locations, such as an elevator.

Other Uses of Presence Detection

Radar system 300 and depth sensors, as described above can have manyother uses. For example radar system 300 and depth cameras 1202 may beused to monitor the presence of persons in a conference room or office,which can be useful for HVAC or lighting controls, by activating suchsystems only when persons are detected in the area. If radar system 300is used, it can be positioned outside of a room to let a person outsideby use of an indicator, such as a sign or light, know if the room iscurrently unoccupied and available, or if someone is in the room.Alternatively, radar system 300 directed out of a room, such as anoffice or meeting room, can let a person in the room know if a person iswaiting outside without a disruption occurring. Similarly radar system300 or depth cameras 1202 could be used to determine if a bathroom isoccupied and alert a person outside.

Portable units with radar systems 300 could be used for search andrescue, military and police purposes. Radar systems 300 can be used todetermine the location of a person needing assistance or posing a threatfrom outside a room or building.

Another use case occurs when determining if a person has been leftbehind, for example should an emergency occur in an office, or in abiohazard room, radar system 300 or a depth cameras 1202 can be used todetermine if a person is still present in the affected area. Inparticular as radar signals pass through walls, radar system 300 can bepositioned to determine if a person is within a room from a locationoutside the room. For example a radar system 300 can be positionedstrategically to monitor a floor of an office building and be configuredto quickly relay to rescue personnel whether a person is still breathingand where the person is on the floor.

The ability to detect whether a person is sleeping or not can be used inseveral different contexts. For example, hospital beds can be monitoredto determine whether or not a patient is sleeping. Prison cells can bemonitored to determine if a prisoner is awake or active when they shouldnot be. Sleeping persons could be detected in alleys, subways or railcars. Other uses would be elderly care rooms, bathrooms in hospitals orelderly care facilities, or even closets.

Other Uses of Radar Systems

The fact that radar signals pass through walls allows radar system 300to have a number of uses in multi-room environments to detect thepresence or absence of persons in one or more rooms, and if a person ispresent, determine the length of time the person has been present. Radarsystem 300 can also be used to detect the level of activity of a person,for example if they are sleeping or moving.

One such multi room environment is a retail store changing room 2200, asshown in FIG. 22. One radar system 300 may be positioned to monitorseveral changing Rooms 01R-12R to detect occupancy. Thus employees coulddetermine if a particular changing Room 01R-12R that was expected to beoccupied is not, or conversely, that a changing Room 01R-12R that wasexpected not to be occupied, is actually occupied.

The occupancy of the Rooms 01R-12R can be displayed to employees using agraph, as shown in FIG. 23. Each Room is shown, and may be listed, orshown as in FIG. 23 from a top down view. Each Room 01R-12R may bemarked, for example as black when empty or a color, such as blue whenoccupied. Other indicators may be used, such as a text string displayindicating whether the room is occupied or unoccupied.

A time limit can be used to determine when the Room has been occupiedlonger than expected and may require assistance from employees. Acolour, for example, red can be used to indicate to a user whether aRoom has been occupied beyond a reasonable time. For example, as shownin FIG. 23, Rooms 02R, 05R, 08R, and 09R are occupied; Rooms 01R, 03R,04R, 06R, 07R, 10R, 11R, and 12R are not. Room 02R has been occupied for5 minutes and 21 seconds; Room 05R, 3 minutes and 42 seconds; and Room09R, 1 minute and 2 seconds; and such Rooms may be colored to show theoccupancy state. Room 08R has been occupied for 13 minutes, 8 secondswhich is over the predetermined time limit, which may be for example, 6,8, 10, 12 or more minutes, and thus may be colored red. When an occupantremains in the Room for longer than the predetermined time, then analert may be generated for staff.

Another multi room environment in which radar system 300 and depthsensors are useful is prisons. As shown in FIGS. 24 and 25, radar system300 can be positioned to monitor both several prison cells 01C to 018Cand the hallway H in between each row of cells 01C-09C and 10C-18C.

A report can be generated, an embodiment of which is partially shown inFIG. 25 and displayed to a user, such as a guard. FIG. 25 shows a tableinclude a column for each cell and the hallway (cells 01C-03C and 18Care shown, and hallway H), Alternatively, a map can be generated of theprison with color codes and descriptions, in a similar fashion asdescribed above with reference to changing rooms (example embodiments ofdescriptions are shown in FIG. 25). In this embodiment the status of theoccupant may be indicated, such as “sleeping” or “active” to let theguard known when a particular cell contains sleeping occupants.Furthermore, radar system 300 or a depth cameras 1202 can be used todetermine if a person is not in their cell, by determining if a cell isunoccupied or if an unauthorized person is in an area (for examplehallway H) in which they should not be present. Radar system 300 ordepth cameras 1202 can also be used to determine is a person is nolonger breathing, for example a person in a cell or hospital bed, andcan provide an alert to obtain assistance should the person stopbreathing.

A display can be presented to an authorized person, for example a prisonguard, so that a quick glance to the display is enough to determine ifanyone is not in a location in which they should be (e.g. missing fromtheir prison cell, or out of a hospital room or care facility bed).Alarms or alerts can be generated should a prisoner (or patient) not bein the correct room. Besides prison cells, the system could be used withdetention cells, holding cells, drunk tanks, and other areas where aperson would be expected or not expected or it is desired to measureduration of a stay. The system could also be used in a farm environment,to allow farmers to ensure animals are present in their appropriatepens, and could be positioned in stables, tie-stalls or barns.

Another use of radar system 300 may be to detect the location ofanimals, such as mice or rats in a home. Small animals can be difficultto detect in a home and are often under floors or within walls. In thisembodiment, radar system 300 may be portable to allow a user to movethroughout a home to locate the animal, and use the ability of radarsignals to pass through walls and floors to indicate the presence of aheartbeat of an animal.

Another use of radar system 300 or depth cameras 1202 could be used iswaiting rooms, such as doctor offices, emergency rooms, and hotel oroffice lobbies to generate an alert if a person is detected.Restaurants, bars, and stock rooms could be monitored by radar system300 or depth cameras 1202 to detect the presence of an unauthorizedperson.

Yet another use of radar system 300 is to monitor transportationfacilities, such as a bus, train or taxi to ensure that no one ispresent.

Radar system 300 or depth cameras 1202

In the embodiments described above more than one presence detectors suchas depth cameras 1202 or radar system 300 may be used.

Other uses for radar system 300 or a depth cameras 1202 may includepeople counting, or wrong way detection (for people or vehicles).

Yet another use for sensor 208 is to determine that a heartbeat is notpresent. For example, a user might believe they are seeing some sort ofapparition, and could use sensor 208 to determine if the object beingviewed has a heartbeat.

While the above description provides examples of the embodiments, itwill be appreciated that some features and/or functions of the describedembodiments are susceptible to modification without departing from thespirit and principles of operation of the described embodiments.Accordingly, what has been described above has been intended to beillustrated non-limiting and it will be understood by persons skilled inthe art that other variants and modifications may be made withoutdeparting from the scope of the invention as defined in the claimsappended hereto.

1. A computer-implemented method for detecting a person in a publiclyaccessible location, comprising: providing a radar system positioned tomonitor the publicly accessible location, the radar system configured totransmit radar signals to and receive radar signals from the location;determining if the radar signals indicate a presence of a person in thelocation; and on determination of the presence of the person, causing analert.
 2. The method of claim 1 wherein the determination of thepresence of a person in the publicly accessible location is triggered bydetection of a heartbeat.
 3. The method of claim 1 wherein thedetermination of the presence of a person in the publicly accessiblelocation is triggered by detection of breathing.
 4. The method of claim1 wherein the radar system is coupled to a ca era.
 5. The method ofclaim 1 wherein the determination of the presence of a person in thepublicly accessible location is triggered by detection of motion.
 6. Themethod of claim 5 wherein the motion is selected from the groupconsisting of: gesturing, bending, stretching and walking.
 7. The methodof claim 4 wherein the camera receives the alert, and if the camera wasnot previously aware of the presence of the person, escalating thealert.
 8. The method of claim 7 wherein the escalation of the alertcomprises including a video stream with an alert.
 9. The method of claim7 wherein the escalation of the alert comprises sending personnel to thepublicly accessible location.
 10. The method of claim 1 wherein thepublicly accessible location is an ATM vestibule.
 11. The method ofclaim 1 wherein the radar system is coupled to an access control system.12. The method of claim 1 wherein the radar system is coupled to anintrusion alarm panel.
 13. A method for controlling loitering in anunsecured location, comprising: providing a radar transceiver in asecured location, the radar transceiver configured to transmit radarsignals to and receive radar signals from the unsecured location;determining if the received radar signals indicate a presence of a firstperson in the unsecured location; if the first person is present,determining if the first person is present in the unsecured location fora predetermined time period; and if the first person is present in theunsecured time period for a time greater than the predetermined timeperiod, providing an alert, report or alarm.
 14. The method of claim 13further determining if the first person is asleep.
 15. The method ofclaim 13 wherein the alert is provided to a second person.
 16. Aphysical security system, comprising: (a) a radar transceiver configuredto monitor a publicly accessible location, the radar system configuredto transmit radar signals to and receive radar signals from the publiclyaccessible location; (b) a processor coupled to the radar transceiverand configured to: (i) determine if the radar signals indicate apresence of a person in the publicly accessible location; (ii) ondetermination of the presence of the immobile person, determine if theimmobile person is present in the unsecured location for a predeterminedtime period; and (iii) on determination of the presence of the personfor a time period greater than the predetermined time period, providinginstructions to cause an alert.
 17. The system of claim 16 wherein theprocessor is further configured to determine if the person is sleeping.18. A method to detect a person loitering in an area under surveillance,comprising: generating a model of the background of the area undersurveillance for a depth sensor; obtaining data maps of the area undersurveillance over a period of time from the depth sensor; comparing thedata maps with the model of the background to generate foreground framesover the period of time; and detecting a person loitering in the areaunder surveillance from analyzing the foreground frames over the periodof time.
 19. The method of claim 18, wherein the model of the backgroundis generated from the data maps received from the depth sensor over asecond period of time when the area under surveillance has no objects inthe field of view of the depth sensor.
 20. The method of claim 18,wherein the length of the period of time is set by a user.